Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance

Introduction A backtest is a historical simulation of an algorithmic investment strategy. Among other things, it computes the series of profits and losses that such strategy would have generated had that algorithm been run over that time period. Popular performance statistics, such as the Sharpe ratio or the Information ratio, are used to quantify the backtested strategy’s return on risk. Investors typically study those backtest statistics and then allocate capital to the best performing scheme. Regarding the measured performance of a backtested strategy, we have to distinguish between two very different readings: in-sample (IS) and outof-sample (OOS). The IS performance is the one simulated over the sample used in the design of the strategy (also known as “learning period” or